Artificial neural networks, or neurocomputers, are biologically inspired; that is, they are composed of elements that perform in a manner analogous to the most elementary functions of the biological neuron. Typically, a neurocomputer is composed of a number (n) of processing elements that may be switches or nonlinear amplifiers. These elements are then organized in a way that may be related to the anatomy of the brain. The configuration of connections, and thus communication routes, between these elements represents the manner in which the neurocomputer will function, analogous to that of a program performed by digital computers. Despite this superficial resemblance, artificial neural networks exhibit a surprising number of the brain's characteristics. For example, they learn from experience, generalize from previous examples to new ones, and abstract essential characteristics from inputs containing irrelevant data. Unlike a von Neumann computer, the neurocomputer does not execute a list of commands (a program). Rather, the neurocomputer performs pattern recognition and associative recall via self-organization of connections between elements.
Artificial neural networks can modify their behavior in response to their environment. Shown a set of inputs (perhaps with desired outputs), they self-adjust to produce consistent responses. A network is trained so that application of a set of inputs produces the desired (or at least consistent) set of outputs. Each such input (or output) set is referred to as a vector. Training is accomplished by sequentially applying input vectors, while adjusting network weights according to a predetermined procedure. During training, the network weights gradually converge to values such that each input vector produces the desired output vector.
Because of their ability to simulate the apparently oscillatory nature of brain neurons, oscillatory neurocomputers are among the more promising types of neurocomputers. Simply stated, the elements of an oscillatory neurocomputer consist of oscillators rather than amplifiers or switches. Oscillators are mechanical, chemical or electronic devices that are described by an oscillatory signal (periodic, quasi-periodic, almost periodic function, etc.). Usually the output is a scalar function of the form V(ωt+φ) where V is a fixed wave form (sinusoid, saw-tooth or square wave), ω is the frequency of oscillation, and φ is the phase deviation (lag or lead).
Recurrent neural networks have feedback paths from their outputs back to their inputs. As such, the response of such networks is dynamic in that after applying a new input, the output is calculated and fed back to modify the input. The output is then recalculated, and the process is repeated again and again. Ideally, successive iterations produce smaller and smaller output changes until eventually the outputs become constant. To properly exhibit associative and recognition properties, neural networks, such as is required by Hopfield's network, must have a fully connected synaptic matrix. That is, to function optimally, recurrent network processing elements must communicate data to each other. Although some prototypes have been built, the commercial manufacture of such neurocomputers faces a major problem: A conventional recurrent neurocomputer consisting of n processing elements requires n2 connective junctions to be fully effective. The terms connector or connective junction, as used herein throughout, are defined as a connective element that enables one processing element to receive as input data output data produced by itself or any other one processing element. For large n this is difficult and expensive.
Accordingly, a need exists for a neurocomputer with fully recurrent capabilities and requiring a minimal number of connective devices between processing elements.